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Abstract 


In  this  report,  an  overview  of  information  fusion  techniques  for  dependent  and  independent 
sources,  specifically  for  biometric  applications,  is  provided.  The  information  fusion 
architecture  is  presented  for  both  dependent  and  independent  sources  addressing  in  detail  the 
various  fusion  techniques  at  four  different  levels  namely:  raw  data  or  signal  level,  feature 
level,  decision  level  and  multi-level  integrated  fusion.  Furthermore,  the  report  addresses  the 
question  of  whether  independent  biometric  sources  can  be  fused  to  provide  multi-modal 
biometric  system  with  enhanced  performance.  The  report  shows  that  even  when  the  sources 
are  independent,  the  performance  of  a  multi-modal  biometric  system  can  be  better  than  that  of 
a  biometric  system  based  on  single  source.  The  performance  is  measured  in  terms  of  total 
false  accept  rate  (FAR)  and  false  rejection  rate  (FRR).  The  conditions  for  achieving  an 
improved  performance  for  the  decision  level  fusion  using  AND,  OR  and  majority  voting  are 
derived  theoretically  and  confirmed  through  computer  simulations. 


Keywords:  information  fusion,  biometrics,  dependent,  independent,  signal  fusion,  feature 
fusion,  decision  fusion. 
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Executive  summary 


Information  fusion  utilizes  a  combination  of  different  sources  of  information,  either  to 
generate  one  representational  format,  or  to  reach  a  decision.  Information  can  be  from  both 
dependent  and  independent  sources.  In  this  report,  a  literature  review  of  information  fusion 
techniques,  in  particular  for  biometric  information  fusion,  is  given.  We  classify  fusion  into 
two  main  categories:  fusion  of  dependent  sources  and  fusion  of  independent  sources.  Both 
categories  of  fusion  can  be  carried  out  in  four  different  levels:  raw  data  level,  feature  level, 
decision  level  and  multi-level  integrated  fusion.  For  the  dependent  sources,  raw  data  level 
fusion  is  widely  applied  particularly  in  target-tracking,  remote  sensing  image  fusion, 
navigation,  etc.  However,  it  is  not  suitable  for  independent  biometric  sources  since  the 
biometric  traits  contain  different  signal  formats.  Feature  level  fusion  is  applicable  to  both 
independent  and  dependent  sources.  Developed  methods  include:  nonparametric  techniques 
(nearest-neighbor  rule,  Parzen  window  and  metric  classifier),  soft  computing  techniques 
(MLP,  RBF,  SVM,  GA,  and  fuzzy  logic),  unsupervised  learning  and  clustering  techniques  and 
stochastic  techniques  (Bayesian  classifier,  HMM  and  EM  ML).  Since  features  from  different 
sources  are  concatenated  resulting  in  a  vector  with  large  dimensions,  the  problem  of  high 
dimensionality  must  be  addressed.  Features  can  also  be  used  in  an  individual  classifier,  which 
outputs  matching  scores.  In  this  scheme,  rule  based  methods  (majority  voting,  aggregation 
rules,  Borda  count)  are  in  the  mainstream  while  others  such  as  statistical  decision  (Bayesian 
decision,  DM),  information  based  (expert  system,  fuzzy  sets),  and  learning  based  (SVM, 
MLP,  RBF)  methods  have  also  been  proposed.  Since  in  the  case  of  decision  level  fusion, 
individual  classifiers  output  a  hard  decision  it  is  favorable  to  the  independent  sources  while 
some  literature  exist  that  formulates  decision  level  fusion  for  dependent  sources  as  well.  The 
techniques  here  include  rule-based  (majority  voting,  AND/OR,  PROD/SUM,  etc),  machine 
learning  and  NN,  linear  classifier,  statistical  decision  theory  (ML,  Bayesian,  DM),  fuzzy  k- 
means,  logistic  regression  etc.  Among  them,  the  SVM  is  reported  to  have  a  better 
performance  while  the  rule-based  algorithms  are  the  simplest  and  easiest  to  analyze. 

Preliminary  theoretical  analysis  is  developed  here  using  decision  level  fusion  to  determine 
whether  fusion  of  independent  biometric  sources  could  improve  the  overall  identification 
performance.  The  performance  is  evaluated  by  two  common  indices:  false  acceptance  rate 
(FAR)  and  false  rejection  rate  (FRR)  as  well  as  the  total  error.  When  two  independent 
biometric  traits  are  used,  the  performance  of  the  integrated  system  based  on  the  AND/OR 
fusion  rules  is  proved  to  give  better  performance  in  terms  of  total  error  than  that  of  an 
individual  biometrics  under  certain  condition.  FAR  and  FRR  decreases  with  number  of 
biometric  sources.  However,  the  FAR  and  FRR  cannot  be  simultaneously  reduced.  To  this 
extent,  we  also  investigate  the  choice  of  majority  voting  rule  for  three  independent  source 
fusions  in  this  report.  Fusion  of  independent  sources  is  found  to  produce  improved 
performance  than  those  of  AND/OR  and  individual  sources  under  certain  conditions. 
Computer  experiments  are  carried  out  to  confirm  the  theoretical  derivation  and  to  illustrate 
that  that  fusion  of  independent  sources  is  indeed  possible  and  results  in  a  better  overall 
performance  for  biometric  identification. 


Li,  W.,  Huang,  D.,  Leung,  H.  2005.  Fusion  of  Dependent  and  Independent  Biometric 
Information  Sources.  DRDC  Ottawa  CR  2005-052.  University  of  Calgary. 
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1.  Introduction 


Information  fusion  is  usually  considered  to  be  combining  various  sources  of  information,  either  to 
generate  one  representational  format,  or  to  reach  a  decision.  The  motivations  for  using 
information  fusion  include:  (1)  Utilizing  multi-sensor  fusion  to  increase  the  estimation  accuracy 
of  target-tracking  [1-4];  (2)  Utilizing  complementary  information  to  reduce  the  measurement 
errors  [5-7];  (3)  Utilizing  multiple  classifier  fusion  to  increase  the  correct  classification  rates  [5- 
16];  (4)  Reducing  the  cost  of  implementation  possibly  by  using  several  cheap  sensors  rather  than 
one  expensive  sensor. 

For  the  purpose  of  this  study,  we  divide  information  fusion  into  two  categories:  dependent  and 
independent  source  fusion.  Fusion  under  both  categories  can  be  carried  out  at  four  different 
levels:  raw  data-level  fusion  [22-28],  feature-level  fusion  [29-57],  decision-level  fusion  [58-73], 
and  multi-level  integrated  fusion  [74-77].  Information  fusion  has  a  wide  range  of  applications 
including  (1)  in  defense  systems  for  target  detection,  identification,  surveillance,  tracking,  and 
threat  assessment;  (2)  in  geoscience  and  Geomatics  for  vehicle  localization  and  navigation, 
segmentation  and  classification  of  remote  sensing  images;  (3)  in  robotics  and  intelligent  vehicles 
that  require  identification  of  environments  and  navigation;  (4)  in  medicine  where  information 
fusion  is  used  for  diagnosis  and  modeling  of  the  human  body,  classification  of  different  tissues, 
and  3D  imaging;  (5)  in  industrial  engineering. 

Multi-modal  biometric  system  is  a  relatively  new  application  of  information  fusion  while 
individual  biometrics  has  been  used  for  a  fairly  long  time.  For  example,  fingerprint  has  been 
widely  used  by  police  for  person  verification  and  identification.  To  increase  the  reliability, 
biometric  fusion  especially  multi-modal  biometric  fusion  has  drawn  a  lot  of  attention  recently. 
Common  biometrics  include  fingerprint  [17],  face  [18],  hand  geometry,  finger  geometry,  iris, 
retina,  signature,  voice,  gait,  smell,  keystroke,  ECG,  etc  [20,  21],  While  unimodal  biometrics  uses 
the  fusion  of  multiple  measurements,  it  can  be  considered  as  the  fusion  of  dependent  sources.  In 
the  case  of  multi-modal  biometric  fusion  such  as  fingerprint  and  face,  the  information  fusion  is 
performed  over  independent  sources  since  the  two  sources  hardly  have  any  correlation  in  the 
statistical  sense.  Although  many  unimodal  and  multi-modal  biometric  fusion  techniques  have 
been  proposed  in  the  literature,  theoretical  analysis  of  these  fusion  methods  has  not  been 
addressed  yet.  In  particular,  very  few  published  papers  are  found  that  have  rigorous  results  on 
multi-modal  biometrics  [81-83]. 

There  are  two  main  objectives  of  this  report.  First,  an  overview  of  information  fusion  techniques 
for  dependent  and  independent  sources,  especially  for  the  biometrics  applications  will  be  given. 
Various  fusion  techniques  for  dependent  and  independent  sources  are  reviewed  and  briefly 
explained  in  the  report.  Second,  we  will  address  the  question  of  whether  independent  biometric 
sources  can  be  fused  to  provide  a  multi-modal  biometric  system  with  enhanced  performance.  The 
rest  of  this  report  is  organized  as  follows.  Section  2  introduces  the  architecture  of  information 
fusion.  Dependent  and  independent  sources  based  information  fusion  techniques  are  overviewed 
in  Section  3  and  Section  4  respectively.  Section  5  presents  the  theoretical  analysis  of  decision- 
level  fusion.  Simulation  results  are  presented  in  Section  6.  Finally,  conclusions  are  drawn  in 
Section  7. 
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2.  The  Structure  of  Information  Fusion 


2.1  Comparison  of  fusion,  integration,  and  classification 


Before  discussing  the  structure  of  information  fusion,  we  first  compare  the  concepts  of  fusion, 
integration,  and  classification.  Fusion  and  classification  are  compared  in  Figure  1.  There  are  many 
kinds  of  information  fusion  applications  such  as  multi-sensor  fusion  based  target-tracking, 
navigation,  image  fusion,  and  classifier  fusion.  Classification  can  be  divided  into  two  categories: 
non-fusion  based  classifier  and  fusion  based  classifier.  For  classification  problems,  there  is  a 
common  part  for  fusion  and  classification,  i.  e,  classifier  fusion.  Classifier  fusion  is  also  called 
classifier  combination  by  some  researchers. 


Figure  1.  Relationship  between  fusion  and  classification  problems 


There  is  no  consistent  definition  for  multi-sensor  integration  in  the  literature.  One  definition  is 
given  in  [4].  Multi-sensor  fusion  is  one  component  of  multi-sensor  integration.  Besides  multi¬ 
sensor  fusion,  multi-sensor  integration  also  includes  sensor  registration,  sensor  modeling,  and 
other  functional  components. 


2.2  Classification  of  information  fusion  techniques 


According  to  the  relationship  of  the  sources  to  be  fused,  information  fusion  can  be  divided  into 
two  categories:  dependent  sources  based  and  independent  sources  based.  Each  category  can  be 
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further  divided  into  four  subcategories:  raw  sensor  measurement  level  (sensor-level),  feature- 
level,  decision-level,  and  multi-level  integrated  fusion  as  shown  in  Figures  2. 


Figure  2.  Classification  of  information  fusion  techniques 


The  structure  of  information  fusion  is  shown  in  Figure  3.  In  the  raw  data- level  fusion,  the  raw 
data  from  multiple  sensors  are  combined  directly.  For  example,  multiple  remote  sensing  images 
are  fused  together  to  produce  one  single  image  by  averaging  the  intensities  of  multiple  images. 
Random  measurement  noise  of  fingerprint  images  can  be  reduced  by  combining  the  raw  images. 
After  raw  data  fusion,  feature  is  then  extracted  from  the  fused  data  by  feature  extraction  module 
(FEM),  and  finally  processed  by  the  matching  module  (MM)  and  decision  module  (DM). 

The  second  type  is  feature-based.  There  are  two  kinds  of  feature-based  information  fusion 
approaches.  The  first  is  feature  concatenation.  The  feature  vectors  from  multiple  sensors  are 
concatenated  to  form  a  new  feature  vector  with  a  larger  dimension.  The  concatenated  feature 
vector  is  then  input  into  the  matching  module  and  decision  module.  The  second  is  called  score- 
level  fusion  or  classifier  fusion  by  some  researchers.  The  individual  feature  vectors  from  each 
sensor  are  first  input  into  their  own  feature  matching  modules  (FMM).  Each  FMM  provides  a 
matching  score  indicating  the  proximity  of  the  feature  vector  with  the  template  vector.  These 
scores  are  finally  combined  to  assert  the  veracity  of  the  claimed  identity. 

The  third  type  is  decision-based.  For  each  sensor,  there  is  a  decision  making  module  that 
classifies  its  own  extracted  feature  vector  into  two  classes  -  accept  or  reject.  The  decision  results 
(accept/reject)  from  multiple  sensors  can  be  fused  finally  by  some  methods  such  as  the  majority 
voting  scheme. 


DRDC  Ottawa  CR  2005-052 


3 


Figure  3.  Different  levels  of  information  fusion  (FM:  fusion  module ,  MM:  matching  module ,  DM:  decision 
module) 
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Accept/Reject 

Figure  4.  The  multi-level  integrated  fusion  structure 

Some  researchers  have  proposed  multi-level  integrated  schemes  for  information  fusion.  Three- 
level  integrated  information  fusion  for  classification  problems  is  shown  in  Figure  4.  One  classifier 
utilizes  the  raw  data  from  multiple  sensors  directly  for  classification.  Another  classifier  utilizes 
the  features  from  multiple  sensors  for  classification.  The  outputs  from  raw  data-level  and  feature- 
level  fusion  are  then  input  into  the  decision  fusion  module  to  make  a  final  decision  -  accept  or 
reject. 
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3.  Information  Fusion  for  Dependent  Sources 

3.1  Raw  data  level 


Different  applications  require  different  levels  of  information  fusion.  Raw  data-level  fusion  has 
been  widely  applied  for  multi-sensor  fusion  in  target-tracking,  remote  sensing  image  fusion, 
medical  image  fusion,  GPS/INS  navigation,  etc.  Although  raw  data-level  fusion  can  also  applied 
to  biometric  fusion,  it  is  a  less  popular  method  compared  with  feature-level  and  decision-level 
fusions.  In  this  section,  raw  data-level  fusion  techniques  for  non-biometric  applications  are 
reviewed.  Then  biometric  fusion  techniques  are  introduced. 

Compared  with  biometric  fusion,  multi-sensor  fusion  has  been  employed  for  many  applications 
including  the  most  popular  target-tracking  application.  Many  multi-sensor  fusion  based  target¬ 
tracking  methods  have  been  elaborated  in  [3].  The  most  widely  used  one  among  them  is  the 
extended  Kalman  filter  (EKF)  method,  which  utilizes  the  raw  measurements  of  multiple  sensors 
to  estimate  the  states  of  the  target.  The  second  important  application  of  non-biometric  fusion  is 
navigation.  A  comprehensive  review  of  various  methods  to  fuse  GPS  and  INS  sensors  was  given 
in  [22].  The  fusion  techniques  were  divided  into  three  categories  as:  (1)  Probabilistic  model;  (2) 
Least-squares  techniques:  EKF,  weighted  EKF,  fuzzy  EKF,  adaptive  EKF  [24],  and  optimal 
theory;  (3)  Intelligent  fusion:  fuzzy  logic  [26],  neural  network  [23],  and  genetic  algorithms.  An 
optimal  nonlinear  filtering  to  integrate  GPS  and  INS  sensors  was  utilized  in  [25],  while  a 
constrained  unscented  Kalman  filter  (UKF)  was  used  in  [27]  to  fuse  GPS,  INS,  and  digital  map 
information  for  vehicle  localization.  An  UKF  method  to  register  and  fuse  dissimilar  sensors  was 
proposed  in  [28]  for  cooperative  driving  application.  Although  multiple  dissimilar  sensors  have 
been  fused  for  tracking  and  navigation  applications,  the  measurements  are  based  on  the  same 
source  and  hence  they  can  be  considered  as  dependent  sources  based  fusion.  Some  dynamic 
modeling  methods,  such  as  gait  recognition,  have  been  proposed  for  biometrics  and  hence  EKF 
and  UKF  based  methods  can  be  used  for  biometric  fusion. 

A  biometric  system  is  essentially  a  pattern-recognition  system  that  recognizes  a  person  based  on  a 
feature  vector  derived  from  specific  physiological  or  behavior  characteristic  that  the  person 
possesses  [13].  Biometric  system  can  be  operated  in  two  modes:  verification  and  identification. 
The  commonly  used  biometrics  include  fingerprint,  face,  hand  geometry,  finger  geometry,  iris, 
voice,  palmprint,  DNA,  ECG,  signature,  etc.  The  comparison  of  various  biometric  techniques  is 
given  in  [13]  and  is  presented  in  Table  1.  Since,  each  biometric  technique  has  its  own  limitations 
and  advantages,  multi-modal  biometric  fusion  for  improved  performance  is  gaining  popularity.  A 
comprehensive  overview  of  different  multi-modal  biometric  fusion  techniques  is  given  in  [12-16]. 
Raw  data  fusion  can  be  used  for  unimodal  biometric  fusion.  For  example,  multiple  fingerprint 
images  can  be  obtained  from  different  sensors.  Averaging  of  these  fingerprint  images  can  reduce 
the  random  measurement  noise. 
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Table  1.  Comparison  of  various  biometric  technologies  [2] 


Biometric 

characteristic 

3 

- 

>■ 

/ 

jU 

> 

7 

O 

c 

s 

u 

w 

>. 

v 

4# 

V 

u 

0# 

w 

y 

e 

5 

> 

w 

Facial  thermogram 

H 

ii 

L 

ii 

M 

ii 

L 

1  land  vein 

VI 

M 

M 

M 

M 

M 

L 

(■ait 

M 

L 

L 

11 

L 

H 

M 

ke\ stroke 

L 

L 

L 

M 

L 

M 

M 

Odor 

H 

H 

H 

L 

L 

M 

L 

Ear 

M 

M 

H 

M 

M 

H 

M 

1  land  geometry 

M 

M 

M 

H 

M 

M 

M 

Fingerprint 

M 

H 

11 

M 

11 

M 

M 

Face 

H 

L 

M 

H 

L 

H 

II 

Retina 

H 

H 

M 

L 

H 

L 

lri> 

H 

H 

H 

M 

H 

L 

L 

Palm  print 

VI 

H 

II 

M 

11 

M 

M 

Voice 

VI 

L 

L 

M 

L 

H 

II 

Signature 

L 

L 

L 

H 

L 

H 

H 

DMA 

H 

H 

H 

L 

H 

L 

Legend:  H  -  High  M  -  Medium  L  -  Low 


3.2  Feature-Level 

Feature-level  fusion  is  the  second  category  of  information  fusion.  Based  on  different  combination 
of  features,  we  can  divide  it  into  two  schemes  as: 

1.  Features  from  multiple  sensors  are  concatenated  to  form  a  new  feature  vector  with  larger 
dimension.  The  concatenated  feature  vector  is  input  into  a  feature  matching  module  and 
decision  module  for  decision  making. 

2.  Each  feature  vector  from  individual  sensor  is  input  into  a  feature  matching  module  first. 
The  outputs  (i.e.,  matching  scores)  of  the  feature  matching  modules  are  fused  through 
some  methods.  Feature-level  fusion  has  been  widely  used  for  biometric  fusion  and  target 
identification.  We  mainly  discuss  the  biometric  fusion  problem  here. 


3.2.1  Fusion  at  the  feature  extraction  level 

The  data  obtained  from  each  sensor  and  the  same  biometrics  is  used  to  compute  a  feature  vector. 
Feature  vectors  from  different  sensors  are  concatenated  together  to  form  a  single  new  feature 
vector.  PCA  and  LCA  may  be  required  to  reduce  the  dimension  of  the  concatenated  feature  vector 
before  input  into  a  classifier.  Target  recognition  and  biometric  verification  and  identification 
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problems  can  be  considered  as  a  classification  problem.  The  following  classifiers  can  be  used 
based  on  the  concatenated  feature  vector  [29]: 

•  Nonparametric  techniques: 

o  Nearest-Neighbor  rule,  k-Nearest-Neighbor  rule,  Parzen  window,  Metrics 
classifier 

•  Soft  computing  techniques: 

o  MLP,  Recurrent  NN,  RBF,  SVM,  Probabilistic  NN,  Fuzzy  NN,  ANFIS,  GA 

•  Stochastic  methods: 

o  Bayesian  classifier,  Maximum  likelihood.  Component  analysis,  Expectation- 
maximization,  Flidden  Markov  Model 

•  Non-metric  based  methods: 

o  Decision  trees,  CART  (classification  and  regression  trees),  Recognition  with 
strings,  Grammatical  methods.  Rule-based  methods 

•  Unsupervised  learning  and  clustering: 

o  FCM,  Unsupervised  Bayesian  classifier,  Graph-theoretic  methods. 

Compared  to  the  classifier  fusion  level,  concatenated  feature  scheme  is  less  popular.  We  found 
only  two  references  that  employ  this  approach  to  multimodal  biometric  fusion  for  independent 
sources  and  no  reference  was  found  for  dependent  sources  based  biometrics  fusion. 

3.2.2  Classifier  fusion  level 

Classifier  fusion  and  decision  fusion  are  two  of  the  most  popular  schemes  for  pattern  recognition 
problems  such  as  target  classification  in  military  applications  and  biometric  verification  and 
identification.  Dependent  sources  based  classifier  fusion  techniques  can  be  categorized  as: 

•  Rule  based  algorithms: 

o  Majority  voting  [9,  31,  33] 

o  Aggregation  rules:  minimum,  maximum,  averaging  (simple  averaging,  optimal 
averaging  [31],  product  [34],  logistic  rule  [38],  Borda  count  [9,  35] 

•  Statistical  decision  based  algorithms: 

o  Bayesian  estimation  [4] 
o  Dempster-Shafer  (DS)  evidential  reasoning  [9] 
o  Non-parametric  density  estimation  [9] 

•  Information  based  algorithms: 

o  Expert  systems  [34] 
o  Combination  by  fuzzy  integral  [3] 
o  Fuzzy  sets  and  possibility  theory  [36] 

•  Learning-based  fusion: 

o  SVM,  feed  forward  neural  network,  MLP,  RBF  [9] 

•  Others: 

o  Linear  regression  [33] 
o  Logistic  transform  [9,  37]. 

A  Bayesian  frame  and  DS  theory  was  used  in  [31]  to  fuse  Bayesian  classifier,  k-NN,  and 
distance-based  classifiers  for  handwriting  recognition.  Using  some  individual  classifier, 
handwriting  words  were  first  recognized.  Then  the  scores  of  the  individual  classifiers  were 
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combined  by  the  Bayesian  and  DS  methods.  Their  experiments  showed  that  the  correct 
recognition  rates  could  be  improved  by  about  5%  compared  with  the  individual  classifiers.  A 
method  for  fusing  multiple  instances  of  biometric  data  to  improve  the  performance  of  a  personal 
verification  system  was  proposed  in  [33].  The  fusion  problem  was  formulated  in  the  framework  of 
Bayesian  estimation  theory.  Experimental  studies  on  the  M2VTS  database  [32]  showed  that  a 
reduction  in  error  rates  is  up  to  about  40%.  Four  combination  strategies  are  [32]: 


Averaging 


A  j= 1 

(1) 

Max  rule 

P{ Wj  jc)  =  max^L,  P(wj  |jc) 

(2) 

Min  rule 

P(Wj  |x)  =  minj=1  P(wj\x) 

(3) 

Median  rule 

P( Wj  jc)  =  median J=l  P(wj  |jc) 

(4) 

Expert  fusion  strategies,  including  product,  min,  max,  median  and  vote,  and  single  expert  scheme 
were  compared  in  [34].  It  was  also  shown  that  there  exists  a  noise  level  boundary  across  which 
the  relative  performance  of  classifier  combination  rules  changes.  For  expert  fusion  techniques 
affected  by  uniform  noise,  it  was  found  that  the  minimum  and  product  rules  were  best  for  noise 
less  than  a  particular  threshold.  When  the  noise  level  was  greater  than  the  threshold,  the 
performance  of  these  two  rules  degraded  and  the  sum  rule  became  the  best  one. 

Borda  count  was  used  in  [35]  to  fuse  multiple  algorithms  for  word  recognition.  A  novel  Borda 
count  for  fusion  based  on  ranks  and  confidence  was  proposed.  All  experiments  were  performed 
on  real-world  handwritten  words  taken  from  the  CEDAR  benchmark  database.  The  word 
recognition  results  were  the  highest  (91%)  among  published  results  for  handwritten  words  (before 
2001).  The  proposed  technique  was  shown  in  Figure  5. 
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Figure  5.  The  Borda  count  method  for  handwritten  recognition  [35], 


A  method  to  fuse  multiple  handwritten  word  classifiers  based  on  data-dependent  densities  in  a 
Choquet  fuzzy  integral  was  proposed  in  [36]  that  outperforms  neural  networks,  Borda  count, 
weighted  Borda  counts,  and  Sugeno  fuzzy  integral  (about  2%  improvement).  The  Choquet  fuzzy 
integral  of  the  function  h  with  respect  to  a  fuzzy  measure  g  is  defined  as  follows: 

e  =  Y[h{xi  )-h{xi  A  )]g",  (5) 

i=i 


where 

h(x0)=0,  h(x1)<h(x2)<A  <  h(xn)  , 

and  „  =  \g({xl,xM,A  ,A  Xj}),  i  <  j 

I  0,  otherwise. 

When  fuzzy  integral  is  applied  to  classifier  fusion,  every  classifier  produces  a  confidence  value 
for  each  class.  These  confidence  values  are  represented  by  the  function  h.  The  overall  confidence 
for  that  class  is  the  fuzzy  integral  value.  The  class  with  the  largest  integral  value  can  be  taken  as 
the  final  decision  if  a  crisp  decision  is  needed. 

Fingerprint  matching  algorithms  are  often  based  on  different  representations  of  the  input 
fingerprints  and  hence  complement  each  other.  A  logistic  transform  was  used  in  [37]  to  integrate 
the  output  scores  from  three  different  fingerprint  matching  algorithms.  Let  /,  (x[ )  and  6',  (x, )  be 
the  imposter  and  genuine  distribution  of  the  ilh  matcher,  i  =  1,2.  The  logistic  function  was  used  to 
map  the  output  scores  of  these  two  matching  algorithms  into  a  single  score.  The  integration 
scheme  is  shown  in  Figure  6.  An  experimental  result  is  given  in  Figure  7. 
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Figure  6.  Integration  of  two  fingerprint  matching  algorithms  using  the  logistic  transform 


Figure  7.  ROC  curves  of  integration  of  algorithms  A  and  B 


A  logistic  transform  method  similar  to  [37]  was  proposed  in  [38]  to  fuse  optimal  and  capacitive 
sensors  for  fingerprint  verification.  The  logistic  transformation  function  is  defined  as 


1  +  exp[-(w0  +  uvs’o  +  w2scy\ 
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The  ROC  curves  of  the  fusion  methods  are  shown  in  Figure  8.  The  performance  of  the  fusion 
method  is  better  than  the  performance  of  the  single  classifiers. 


Figure  8.  ROC  curves  of  the  optical  sensor  and  two  fusion  methods 


A  multi-channel  approach  to  fingerprint  identification  was  proposed  in  [39].  This  method  was 
tested  on  4000  images  in  the  NIST4  database.  A  two-stage  classifier  method  is  proposed.  This 
method  uses  K-nearest  neighbor  classifiers  in  the  first  stage  and  a  set  of  neural  network  classifiers 
in  the  second  stage  to  classify  a  feature  vector  into  one  of  the  five  classes.  The  classification 
accuracy  was  improved  from  94.8%  to  96%  for  the  five-class  classification  problem. 

Flighest  rank,  Borda  count,  and  linear  regression  were  used  in  [40]  for  word  recognition.  The 
performance  of  different  methods  was  compared  in  Table  2.  The  recognition  rates  are  improved 
greatly  by  the  fusion  method. 
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Table  2.  Performance  comparison  of  character  classifiers 


Correct  Rate  (Vf )  at  Top  X 

(Choices 

Model 

C.'lassitiertsl 

1 

2 

3 

4 

5 

10 

PBC 

79.3 

87.8 

91.2 

92.8 

94.3 

97.6 

PNC 

H5.3 

91.3 

93.3 

94.7 

95.4 

97.7 

P2N 

85.8 

91.9 

93  9 

94.9 

95.7 

97  9 

BBC 

79.1 

87.5 

907 

92  4 

93,9 

97.1 

BNC 

84.3 

90.8 

93.3 

94.6 

95.4 

97.7 

B2N 

85.4 

91.7 

93.9 

95.1 

95.7 

98.0 

Logistic  Regression 

1 

PBC.  PNC 

85.4 

92.2 

94.5 

95.7 

96.7 

98.5 

2 

PHI  .  P2N 

86.3 

92.8 

94.  y 

95.9 

96.8 

98.6 

3 

BBC.  BNC 

85.3 

91.8 

94.1 

95.3 

96.1 

98.3 

4 

BBC.  B2N 

860 

92.3 

94.5 

95.7 

96.4 

98.4 

S 

PRC.  BBC 

81.3 

8V.8 

92.7 

94.5 

95.8 

98.3 

6 

PNC.  BNC 

86.7 

92.6 

94.6 

95.6 

96.2 

98.4 

7 

P2N,  B2N 

86.9 

929 

95.0 

95.9 

96.5 

98.5 

8 

PBC,  BBC, 
PNC.  BNC 

88.1 

93.8 

95.7 

96.6 

97.2 

98.8 

9 

PBC.  BC. 
P2N.  B2N 

88.8 

94.1 

95.7 

96.7 

97.4 

98.9 

3.3  Decision-Level 

Very  few  published  papers  have  been  found  for  decision- level  biometric  fusion  based  on 
dependent  sources.  An  optimal  Neyman-Pearson  rule  [59]  was  used  to  combine  multiple 
fingerprint  matching  algorithms  at  the  decision-level.  Experimental  results  conducted  on  a  large 
fingerprint  database  (with  about  2700  fingerprints)  showed  that  the  average  matching 
performance  increased  by  about  3%.  They  further  showed  that  a  combination  of  multiple 
impressions  and  multiple  fingers  improved  the  verification  performance  by  more  than  4%  and 
5%,  respectively. 


This  method  estimates  the  Parzen  window  density  function  based  on  n  observations  given  by 


P(X)  = 


nha  “|(2,r)rf/2|2l1/2 


exp 


2  h 


-{x-xyz-'ix-x.) 


(7) 


Then  the  likelihood  ratio  L  =  P(  X  d  |  M,o )/  P(Xd  |  M,  )  is  used  to  make  the  final  decision  for  a  two- 

class  problem:  Decide  D0  (person  is  an  imposter)  for  high  values  of  T;  Decide  D,  (person  is 
genuine)  for  low  values  of  L. 
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3.4  Multi-Level  integrated  fusion 


A  few  papers  were  published  to  discuss  multi-level  integrated  sensor  fusion.  Multi-level  fusion 
was  utilized  in  [74]  for  target  discrimination,  while  it  was  used  in  [75]  to  gradually  update  the 
performance  of  identity  verification  systems.  A  distributed  data  fusion  structure  was  proposed  in 
[76]  for  autonomous  systems,  while  [77]  used  a  two-level  belief  function  model  to  detect  anti¬ 
personnel  mines.  A  multi-level  data  fusion  for  intelligent  navigation  of  telerobot  was  discussed  in 
[78]. 


14 


DRDC  Ottawa  CR  2005-052 


4.  Information  Fusion  for  Independent  Sources 

4.1  Raw  data-level 


Raw  data-level  fusion  is  not  suitable  for  multi-modal  biometric  fusion  since  the  raw 
measurements  from  different  biometrics  such  as  face  and  fingerprint  cannot  be  combined  to  form 
a  new  measurement. 

4.2  Feature-Level 

4.2.1  Fusion  at  the  feature  extraction  level 

Compared  with  the  score  level  fusion,  concatenated  feature  schemes  are  less  popular.  Only  a  few 
papers  were  published  about  information  fusion  in  the  feature  extraction  level.  A  concatenated 
feature  vector  and  Parzen  classifier  were  used  in  [42]  to  fuse  fingerprint  and  iris  for  improved 
identity  verification.  A  new  feature  vector  x  =  (jct ,  x2 ,  x3 )  was  formulated  based  on  the  feature 
vectors  jc,  (fingerprint),  x2  (iris),  and  x2  (iris  color).  The  classification  rates  of  the  fusion 
method  are  compared  with  single  classifiers  in  Figure  9. 


Figure  9.  Comparison  of  fusion  method  and  single  classifiers 


In  [43],  a  concatenated  feature  scheme  was  proposed  on  fusing  speech  and  face  for  person 
verification.  The  experimental  results  showed  that  for  high  SNRs,  correct  classification  rates  of 
the  fusion  methods  were  higher  than  that  of  the  individual  classifiers.  However,  for  low  SNRs,  the 
correct  classification  rates  of  the  fusion  methods  were  worse  than  the  best  individual  classifier. 
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4.2.2  Classifier  fusion  level 


Independent  sources  based  classifier  fusion  techniques  can  be  classified  as  follows: 

•  Rule  based  algorithms: 

o  Majority  voting  [51,  52,  53] 

o  Aggregation  rules:  sum  [50],  minimum,  maximum,  averaging  (simple  averaging, 
weighted  averaging  [49,  52],  Borda  count  [53],  Mixed  group  ranks  (MGR)  [55] 

•  Statistical  decision  based  algorithms: 

o  Probabilistic  fusion  [52] 
o  Bayesian  estimation  [50,  52,  53] 
o  Dempster-Shafer  evidential  reasoning  [52,  57] 
o  Adaptive  decision  [52] 

•  Information  based  algorithms: 

o  Behavior-knowledge  space  (BKS)  [52] 
o  Combination  by  fuzzy  integral  [53] 
o  Fuzzy  sets  and  possibility  theory  [53] 

•  Learning-based  fusion: 

o  SVM,  Feedforward  neural  network,  MLP,  RBF  [50,  54] 

•  Others: 

o  Combination  by  Zimmermann’s  compensatory  operator  [53] 
o  Score  transformation 

o  Reduced  multivariate  polynomial  model  for  multimodal  biometrics  and 
classification  fusion,  [49,  56] 

Face  and  speech  was  combined  in  [44]  for  person  identification.  The  classifiers  used  include 
SVM,  Bayesian  classifier,  decision  tree,  and  MLP.  Bayesian  method  is  used  to  fuse  the  multiple 
classifiers.  The  ROC  curves  of  the  Bayesian  fusion  and  single  modalities  are  compared  in  Figure 
10.  The  performance  of  identity  verification  is  improved  greatly  by  Bayesian  fusion. 
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Figure  10.  ROC  curves  of  Bayesian  fusion  and  single  modalities 


Figure  11.  ROC  curves  of  fusion,  face,  and  fingerprint 
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Bayesian  statistics  fusion  was  used  in  [45]  for  multi-modal  person  authentication.  HyperBF 
networks  were  used  in  [46]  to  fuse  audio  and  video  features  for  person  identification.  Faces  and 
fingerprints  were  integrated  in  [47]  for  person  identification.  Statistical  estimation  method  was 
used  for  fusion.  The  ROCs  of  the  fusion  method,  face,  and  fingerprint  are  shown  in  Figure  11.  For 
low  acceptance  rates,  the  performance  of  the  fusion  method  is  improved  effectively  than  face-only 
based  and  fingerprint-only  based  methods. 

A  multi-modal  biometric  system  was  introduced  in  [48],  which  integrates  face  recognition, 
fingerprint  verification,  and  speaker  verification  for  person  identification.  This  system  takes 
advantage  of  the  capabilities  of  each  individual  biometrics.  It  can  be  used  to  overcome  some  of 
the  limitations  of  a  single  biometrics.  Preliminary  experimental  results  demonstrate  that  the 
identity  established  by  such  an  integrated  system  is  much  more  reliable  than  the  identity 
established  by  a  single  face  recognition  system,  fingerprint  verification  system,  or  speaker 
verification  system. 

A  reduced  multivariate  polynomial  model  for  classifier  fusion  was  proposed  in  [49].  This  method 
overcomes  the  shortcomings  related  to  complexity  of  conventional  multivariate  polynomial  model 
for  high-dimensional  problems.  The  authors  applied  the  new  model  to  fuse  fingerprint  and  voice 
data  for  person  identification.  The  second-order  (RM2)  and  third-order  (RM3)  multivariate 
polynomial  models  are  compared  with  unimodal  biometric  systems  in  Figure  12.  The 
identification  performance  can  be  improved  greatly  by  the  fusion  methods  for  low  acceptance  rate 
cases. 

SVM  and  sum  rule  for  fusion  of  fingerprint  and  signature  were  proposed  in  [50],  The  output 
scores  of  single  classifiers  are  used  as  the  input  of  the  SVM  network.  The  output  of  the  SVM  is 
the  final  decision  score.  Before  testing,  the  SVM  network  must  be  trained  by  some  training  data 
sets. 


18 


DRDC  Ottawa  CR  2005-052 


Authentic  *cc«pur>c*  rate  (*)  Ai*h»nllc  acceptance  raU  (%) 


(•)  Recaivat  Operating  Curve  (Train  data) 


Figure  12.  ROC  curves  of  second-order  and  third-order  multivariate  polynomial  model  fusion  methods 
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Figure  13.  ROC  curves  of  the  sum  rule  based  fusion  method 
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Figure  14.  ROC  curves  of  the  SVM  method 


Reference  [52]  proposed  decision  templates  for  multiple  classifier  fusion,  and  compared  it  with 
Cl:  MAJ,  majority  voting.  C2:  NB,  Naive  Bayesian;  BKS,  Behavior  knowledge  space  method. 
CC1:  MAX,  maximum  aggregation  rule;  MIN,  minimum  aggregation  rule;  AVR,  average 
aggregation  rule;  PRO,  product  aggregation  rule.  CC2:  PPR,  probabilistic  product;  FI,  fuzzy 
integral.  CI2:  DS,  Dempster-Shafer,  LDC,  Linear  discriminant  classifier  on  the  intermediate- 
output  space;  QDC,  Quadratic  discriminant  classifier;  LOG,  logistic  classifier;  FSH,  Fisher  linear 
classifier.  DT:  decision  templates  with  different  models.  The  comparison  is  shown  in  Table  8  in 
[52]. 

Eleven  classifier  fusion  methods  were  compared  in  [53].  The  comparison  included  majority  rule, 
averaging,  Borda  count,  Bayesian  combination,  weighted  averaging,  combination  by  fuzzy 
integral,  combination  by  fuzzy  integral  with  data-dependent  densities,  combination  by  weighted 
averaging  with  data-dependent  weights,  combination  by  the  BADD  defuzzification  strategy, 
combination  by  Zimmermann’s  compensatory  operator  and  optimizing  the  fuzzy  measure. 

Hyperbolic  function  network,  forward  neural  network,  and  SVM  were  used  in  [54]  for  classifier 
fusion  of  fingerprint  and  face.  They  treated  the  problem  of  combining  fingerprint  and  speech 
biometrics  decisions  as  a  classifier  fusion  problem.  The  output  scores  of  individual  classifiers  are 
used  as  the  input  of  the  neural  networks,  and  the  output  scores  of  the  NN  are  the  final  decision 
scores.  The  different  NN  fusion  methods  can  improve  the  identification  performance  effectively 
as  shown  in  Figure  15.  The  reduced  multivariate  polynomial  model  is  also  used  in  [56]  for  fusion 
of  fingerprint  and  speaker. 


A  mixed  group  rank  (MGR)  method  for  classifier  fusion  was  proposed  in  [55].  A  unifying 
framework  was  proposed  in  this  paper  for  combination  rules  of  rank-based  classifiers.  Borda 
count,  logistic  regression,  and  highest  rank  combination  are  just  special  cases  of  this  framework. 
MGR  improves  the  correct  classification  rates  on  all  tested  data  sets  as  shown  in  Figure  16. 
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Receiver  Operating  Curve  (Train  data) 


Receiver  Operating  Curve  (Test  data) 


Figure  15.  ROCs  of  different  fusion  methods 
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Figure  16.  The  combination  of  three  classifiers  by  MGR  consistently  demonstrates  improved  performance  on 
different  data  sets 
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4.3  Decision-Level 


Decision-level  fusion  can  be  considered  as  a  pattern  classification  problem  such  as  biometrics 
fusion  and  target  recognition.  The  independent  sources  based  decision-level  fusion  schemes 
include: 


•  Rule  based: 

o  Majority  voting  [72],  AND/OR  and  PROD/SUM  rules  [61,  73] 

•  Machine  learning  and  neural  network  [58,  73]: 

o  Expert  [65],  FFNN,  SVM,  MFP  [60],  fuzzy  set  [68],  particle  swarm  [70,  71] 

•  Classifier  based: 

o  Finear  classifier,  Quadratic  classifier,  k-NN  based  classifiers,  decision  trees  [62, 
64,  66] 

•  Statistical  decision  theory  [58,  67]: 

o  Maximum  a  posterior,  maximum  likelihood,  minmax,  Bayesian  [68,  70,  71], 
Dempster-Shafer  (DS)  evidence  theory  [68] 

•  Clustering  for  decision  level  fusion  [69]: 

o  Fuzzy  k-means,  fuzzy  vector  quantization,  median  radial  basis  function 

•  Others: 

o  Fogistic  regression  [58], 

SVM  used  in  [60]  for  decision-level  multi-modal  biometric  fusion.  Face,  fingerprint  and  signature 
were  used  as  the  three  biometric  signals.  Fusion  of  face,  fingerprint,  and  signature  was  considered 
as  a  pattern  classification  problem  in  this  paper,  which  was  implemented  through  SVM.  The  SVM 
classification  problem  can  be  changed  into  a  quadratic  programming  problem  as  shown  in  the 
following  equation 

i=i 

with  y,  (<  W,  0(A,.)  >F  +b)  >  1  -  i  =  1,A  ,/ .  (8) 

£>0,  i=l.  A,/ 

The  classification  scores  of  unimodal  biometrics  are  input  into  the  SVM  network,  and  the  outputs 
of  the  SVM  network  are  the  final  fused  score.  A  training  data  set  is  first  formulated  and  used  to 
train  the  SVM  network.  The  testing  data  set  is  then  used  to  test  the  performance  of  the  SVM 
fusion  method.  The  ROC  curves  of  fusing  different  sets  of  biometrics  by  SVM  are  shown  in 
Figure  17.  When  the  false  acceptance  rate  is  lower  than  20%,  the  fused  results  are  consistently 
better  than  that  of  the  unimodal  biometrics  techniques. 


Minimize  — 
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Figure  17.  ROC  curves  of  SVM  based  multimodal  biometric  fusion 


Voting  was  used  in  [61]  to  increase  the  decision  reliability.  In  majority  voting,  a  consensus  on  the 
decision  is  reached  by  having  a  majority  of  the  classifiers  declaring  the  same  decision.  Decision- 
level  fusion  was  considered  as  a  classifier  problem  in  [62],  It  used  k-NN,  decision  trees,  and 
logistic  regression  in  a  multi-modal  identification  problem.  An  adaptive  model  was  proposed  for 
person  identification  by  combining  speech  and  image  information  [63].  The  flowchart  of  the 
integration  process  is  shown  in  Figure  18.  The  performance  of  the  integration  method  is  compared 
with  face  recognition  model  and  speck  recognition  model.  The  average  recognition  rate  of  the 
integration  method  is  about  91.6%  while  the  recognition  rates  for  the  voice  and  face  models  are 
about  85%  and  79.29%,  respectively. 


Figure  18.  Flowchart  of  the  integration  process  [63] 
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Table  3.  Performance  of  the  speaker  recognition,  face  recognition  and  integration  models 


Person 

Speaker 

recognition 

model 

(%) 

Face 

recognition 

model 

(%) 

Integration 

Model 

(%) 

A 

90 

75 

90 

B 

80 

80 

90 

C 

70 

65 

80 

D 

85 

90 

95 

E 

90 

80 

95 

F 

85 

85 

90 

G 

95 

80 

100 

Average 

85 

79.29 

91.43 

MLP  was  used  in  [64]  to  implement  decision  fusion  using  face,  voice,  and  signature.  The 
classification  scores  of  three  classifiers  for  face,  voice,  and  signature,  are  respectively  used  as  the 
input  of  the  MLP  neural  network,  and  the  outputs  of  the  MLP  are  the  fused  decision  scores. 
Before  testing,  the  MLP  network  must  be  trained  by  some  training  data  sets.  Without  fusion,  the 
recognition  rates  for  voice,  face,  and  signature  are  89%,  99.5%,  and  93%,  respectively.  The 
recognition  rate  with  fusion  is  1 00%. 

Speech  and  face  were  fused  in  [65]  by  expert  systems.  SVM  and  vote  rules  were  used  for  expert 
fusion.  The  authors  investigated  the  behavior  knowledge  space  and  decision  templates  methods  of 
classifier  fusion  in  the  context  of  multi-modal  personal  identity  verification.  The  two  fusion  rules 
have  been  compared  to  simple  combination  rules,  namely  sum  and  vote,  which  do  not  require  any 
training.  Through  extensive  experiments  on  the  XM2VTS  database,  they  found  that  all  these  four 
combination  methods  yielded  performance  improvement.  But  no  strong  evidence  had  been  found 
to  support  the  hypothesis  that  trainable  fusion  strategies  could  offer  better  performance  than 
simple  rules. 

Both  global  and  local  decisions  for  person  identification  based  on  multivariate  polynomial  model 
was  exploited  in  [67].  The  general  multivariate  polynomial  model  can  be  expressed  as 

g(«,  x)  =  £ atx"1  xn2 2  A  x"‘ .  (9) 

i=i 

The  computation  complexity  of  the  model  represented  in  (9)  is  very  high  for  high  dimensional 
data  sets.  So  a  reduced  multivariate  polynomial  model  was  proposed  for  multi-biometric  fusion. 
This  new  model  was  defined  as 
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(10) 


(«,*)  =  «()  +ZZ«#*‘ 

*=1  7=1 
r 

+  E«rf+/(*  1  +  *2  +A  +^/)y 

7=1 
r 

+  ^(ajjt)(x1  +x2  +A  +x,)7  . 

i=2 

Experiments  showed  that  the  fusion  methods  can  improve  the  performance  of  person 
identification  effectively  (as  shown  in  Figure  19). 


(a)  Receiver  Operating  Cnaractenslic  Cumae  (T rain  data) 


Figure  19.  ROC  curves  of  the  fusion  methods 


An  overview  of  different  decision  level  fusion  techniques  including  Bayesian  estimation,  DS,  and 
fuzzy  sets  was  given  in  [68],  Clustering  was  used  in  [69]  for  multi-modal  decision  fusion.  Person 
authentication  results  coming  from  several  modalities  are  combined  by  using  fuzzy  k-means 
(FKM),  fuzzy  vector  quantization  (FVQ),  and  median  radial  basis  function  (MRBF)  network. 
Simulations  showed  that  MRBF  had  better  performance.  Both  [70]  and  [71]  proposed  a  particle 
swarm  optimization  method  to  search  for  the  best  Bayesian  decision  rule  which  was  used  for 
fusing  face,  voice,  and  hand.  The  flowchart  of  this  method  is  shown  in  Figure  20. 
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Figure  20.  Flowchart  of  particle  swarm  optimization  and  Bayesian  fusion 


Face  and  ECG  were  fused  in  [72]  for  person  identification.  Mono-modal  biometric  identification 
systems  exhibit  performance  that  may  not  be  adequate  for  many  security  applications.  Face  and 
fingerprint  modalities  dominate  the  biometric  verification  /  identification  field.  However,  both 
face  and  fingerprint  can  be  compromised  using  counterfeit  credentials.  Previous  research  has 
demonstrated  the  use  of  the  electrocardiogram  (ECG)  as  a  novel  biometric.  This  paper  explored 
the  fusion  of  a  traditional  face  recognition  technique  with  ECG.  System  performance  with  multi¬ 
modality  fusion  can  be  superior  to  reliance  on  a  single  biometric  trait. 


A  multi-modal  approach  for  speaker  verification  was  described  in  [73].  The  system  consists  of 
two  classifiers,  one  using  visual  features  and  the  other  using  acoustic  features.  A  lip  tracker  is 
used  to  extract  visual  information  from  the  speaking  face  which  provides  shape  and  intensity 
features.  They  used  a  weighted  summation  method  to  integrate  the  scores  of  multiple  classifiers. 
The  performance  of  the  integrated  system  outperformed  each  sub-system  and  reduced  the  false 
acceptance  rate  of  the  acoustic  sub-system  from  2.3%  to  0.5%. 
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5.  Theoretical  Analysis  for  Decision  Level  Fusion 


Decision  level  fusion  of  independent  sources  is  considered  here.  For  instance,  two  independent 
sources  are  face  and  voice.  Fusion  scheme  using  these  two  sources  is  denoted  by  A.  Verification 
system  based  only  on  face  is  denoted  by  A1 ,  while  on  voice  by  A2 .  If  A1  provides  95%  correct 
classification  rate  (CCR)  and  A2  provides  only  80%  CCR  individually,  then  it  is  interesting  to 
find  if  A  can  provide  CCR  higher  than  95%,  if  so,  what  is  any  condition  or  what  kind  of  algorithm 
can  reach  this  confidence  level.  If  I  is  an  algorithm,  then  the  task  is  to  find  I  which  acts  on 
independent  sources  so  that  the  output  is  maximized.  This  can  be  written  as 

f  =  maxT^j,  A2) .  (11) 


At  first,  we  give  two  important  performance  indices  in  biometrics  authentication  system.  The  first 
is  false  acceptance  rate  denoted  FAR  hereafter  which  means  wrongly  identifying  an  imposter  to 
be  an  enrollee.  The  other  index  is  false  rejection  rate  denoted  by  FRR  which  means  wrongly 
identifying  an  enrollee  as  an  imposter. 

FAR(t)  =  P(wl\w0)  =  J  ^p{X\wQ)dX  =  \-\  Rop(X\w0)dX,  (12) 

FRR(0  =  T>(w0|w1)  =  J  Rop{X\wx)dX  (13) 

where  w,  denotes  the  genuine  user  while  w0  denotes  the  imposter.  R0  and  Rl  are  two  exclusive 
set  in  the  real  axis.  Both  FAR  and  FRR  are  desirable  to  be  as  low  as  possible  in  authentication 
system.  In  some  systems,  a  low  value  for  FAR  is  emphasized.  Such  systems  include  hank 
authentication  system,  military  access  control  unit,  etc.  For  any  biometrics  authentication  system, 
whatever  classifier  takes,  there  exists  a  great  risk  of  error.  From  the  viewpoint  of  Bayesian 
decision  theory,  this  is  represented  by  the  following  equations  for  a  two  class  problem, 

E(t)  =  Cr  x  FRR(t)  +  Cax  FAR(t) ,  (14) 

Et  (tl )  =  C'  x  FRR '  (tl )  +  C[  x  FAR '  (tl ),  for  i  =  1,  A  ,  N  (15) 


where,  N  is  the  total  sensor  number,  Cr  denotes  the  loss  function  pertinent  to  the  false  rejection, 
and  Ca  denotes  the  loss  function  for  the  false  acceptance.  For  simplicity,  we  assume  that 
Ca  =  C'  and  Cr  =  C' . 
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5.1  Using  AND  rule 


First  we  analyze  the  performance  of  fusing  two  biometric  signals.  In  this  case,  the  relationships  of 
FAR  and  FRR  between  two  individual  classifiers  and  the  integrated  classifier  are  represented  by 
the  following  formulas,  respectively, 


FAR(t)  =  FAR\tl)xFAR\t2),  (16) 

FRR(t)  =  FRR l(tl)  +  FRR 2  (t2 )  -  FRR 1  (tx )  x  FRR 2(t2).  (17) 

Remark  1  Using  an  example  to  explain  (16)  and  (17)  clearly,  we  assume  that 
FAR1  =0.01  and  FAR2  =0.05.  If  there  are  10000  imposters  presented  to  Bx,  then  according  to 
FAR 1  there  will  be  1 00  imposters  passed  by  B] .  Because  of  the  AND  rule,  only  five  imposters  of 
1 00  imposters  can  pass  B2  by  FAR 2 .  Thus  the  false  acceptance  rate  of  the  integrated  system  is 
calculated  by 

Number  of  false  accept  5  ^  0005 

Number  of  imposter  10000 

which  can  be  calculated  from  (16)  as  FAR(t)  =  0.01  *  0.05  =  0.0005 . 


In  a  similar  way,  we  assume  that  FRR1  =  100/1100  =  0.091  and  FRR2  =  50/1000  =  0.05 .  Then  it 
follows  by  the  AND  rule  that  if  1 100  genuine  users  present,  as  a  result  there  are  950  users  passed 
by  the  integrated  system  which  gives  the  false  rejection  rate 

„  Number  of  false  rejections  1100-950  „  ,  „ , 

FRR - — - - - = - =  0.136. 

Number  of  genuine  1100 


It  is  calculated  by  (17),  i.e.,  FRR(t)  =  0.091  +  0.05  -  0.091  x  0.05  =  0.136. 


Lemma  1  If  using  AND  rule  on  the  independent  sources,  then  for  fixed  operating  point  of  B2 ,  the 
integrated  system  can  improve  the  performance  over  the  individual  classifiers  under  the  following 
condition  in  the  sense  of  the  total  error. 

(1  -  kxFAR l(tx))<  FRR  ft)<k2{\-  FAR  ftx))  (18) 


where 


28 


DRDC  Ottawa  CR  2005-052 


(19) 


k  _Ca  1  -FAR2(t2) 

1  Cr  FRR2(t2 ) 

k  _Ca  FAR2(t2) 

2  Cr  \-FRR2(t2)' 

However,  the  integrated  system  will  increases  FRR  while  decreases  the  FAR. 

Proof:  It  is  enough  to  prove  that  E(t)  <  Ef  tf .  This  leads  to  the  following  equations  by 
substituting  (16)  and  (17)  into  E(t), 

C,.FRR 2  (t2  )(1  -  FRR 1  (tx ))  <  CaFAR 1  (?,  )(1  -  FAR 2  (t2  )) ,  (21) 

C,.FRR 1  (t,  )(1  -  FRR 2  (t2 ))  <  CaFAR 2  (t2 )(1  -  FAR 1  (tl ))  .  (22) 

From  (21)  and  (22),  it  can  be  easily  seen  that  to  get  improved  performance,  a  proper  relationship 
should  be  developed  between  the  operating  points  of  two  systems.  In  fact,  at  some  operating 
points,  (21)  and  (22)  will  not  hold,  which  indicates  that  the  performance  cannot  be  improved  for 
the  AND  rule  based  fusion. 

If  the  classifier  B2  operates  at  some  fixed  (FAR2  (t2),  FAR2  (A))  we  can  derive  conditions  for 
Rjfrom  (21)  and  (22).  That  is, 

1  -  kjFAR 1  (q )  <  FRR 1  (A  )  <  k2  (1  -  FAR 1  (tx ))  .  (23) 


From  (16),  it  is  straightforward  that  FAR(t)  <  FAR1  (tx)  and FAR(t)  <  FAR2  (t2) .  This  means  the 
integrated  system  decreases  FAR.  But  from  (17),  FRR(t)  >  FRRl(tx)  and  FRR(t)  >  FRR2  (t2) 
hold.  Thus,  the  FRR  increases. 

Equations  (16)  and  (17)  can  be  easily  generalized  to  the  cased  with  more  than  two  biometrics.  For 
N=  3,  we  have, 

FAR(t)  =  FAR 1  (A  )  x  FRR 2  (t2 )  x  FRR 3  (t3 ) ,  (24) 

FRR(t)  =  FRR 1  (A )  +  FRR 2  (t2 )  +  FRR 3  (t3 ) 

-  (FRR 1  (A )  x  FRR 2  (A)  +  FRR l(tx)x  FRR 3  ( t3 )  +  FRR 2 ( t2 )  x  FRR 3  (t3 )) .  (25) 

+  FRR 1  (t  j )  x  FRR 2  (A )  x  FRR 3  (t3 ) 
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5.2  Using  OR  rule 


For  the  OR  rule,  similar  to  the  AND  rule,  we  can  easily  derive  the  following  relationships  of  FAR 
and  FRR  between  the  individual  classifiers  and  the  integrated  classifier. 

FRR(t)  -  FRR 1  (tj )  x  FRR 2  (t2),  (26) 

FAR(t)  =  FAR 1  (?, )  +  FAR 2  (t2 )  -  FAR 1  (t t )  x  FAR 2  (t2 ) .  (27) 

Therefore,  without  any  further  proof,  we  can  state  the  following  conclusion  similar  to  Lemma  1 . 

Lemma2  If  using  OR  rule  on  the  independent  sources,  then  for  fixed  operating  point  of  B2 ,  the 
integrated  system  can  improve  the  performance  over  the  individual  classifiers  under  the  following 
condition  in  the  sense  of  total  error. 

k2  (1  -  FAR 1  (t, ))  <  FRR 1  (tx )  <  (1  -  klFARl(t1 )) .  (28) 

Flowever,  the  integrated  system  will  increases  FAR  while  decreases  the  FRR. 

Equations  (26)  and  (27)  can  be  generalized  for  the  cases  with  more  biometrics.  For  N=  3,  we 
have 

FRR(t)  =  FRR 1  (fj )  x  FRR 2  (t2 )  x  FRR3  (t3 ) ,  (29) 

FAR(t)  =  FAR 1  (tl )  +  FAR 2  (t2 )  +  FAR 3  (t3 ) 

-  (FAR 1  (tl )  x  FAR2  (t2 )  +  FAR 1  (t1 )  x  FAR 3  (t3 )  +  FAR 2  (t2 )  x  FAR 3  (t3 )) .  (30) 

+  FAR 1  (tt )  x  FAR 2  (t2 )  x  FAR 3  (t3 ) 

5.3  Using  majority  voting  rule 

Assume  that  there  are  N  independent  sources.  For  the  zth  source,  an  individual  classifier  Bl  exists 

with  false  accept  rate  and  false  rejection  rate  [fAR1  (t, ),  FRR'  (f,)} .  If  at  least  ~~~  classifiers 

wrongly  accept  an  imposter  simultaneously,  then  by  the  majority  voting  rule,  the  integrated 
system  will  accept  the  imposter.  Since  each  individual  classifier  is  independent,  the  joint 
probability  of  the  classifiers  is  represented  by 
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(31) 


PU] ,  A  ,  rN  )  =  [FAR 1  (?j )]''  [l  -  FAA 1  (t,  )]  n  [FAR 2  (t2  )}'2  [l  - 
x  A  x  [FAR  n  (tN )]'VI  [l  -  FAR N  (tN )J  rv . 


FAR\t2)  I 


The  probability  of  at  least 


N  +  1 
2 


classifiers  wrongly  identifying  an  imposter  is  then  calculated  by 


FAR(t)=  X  P(rx,F,rN) 

/j  +A  +rN  >(  V+l)  /  2 

^  [par  1  (/,  )]'  [l  -  FAR 1  (A  )J  '  [far  2  (t2  )}2  [l  -  FAR 2  (t2 (32) 

n+A+,w>(jv+i)/2  xA  X  [fAR  n  (tN  )]rwi  [l  -  FAR  N  . 

For  N=  3,  we  have, 


FAR  ( t )  =  FAR 1  (?,  )7v4A  2  (t  2 )  +  AAA  2  (t2  )7v4A 3  (t3 )  -  2ATA 1  (?, )  AAA  2  (t2  )FAR 3  (t3 ) 

=  FAR 1  (?,  )FAA 2  (t2  )(1  -  FAR 3  (t3 ))  +  AAA 1  (t,  )(1  -  AAA 2  (t2  ))FAR 3  (f  3 )  (33) 

+  (1  -  FAA 1  (?,  ))FAR  2  (t2  )FAR 3  (f  3 )  +  AAA 1  (?,  )FAA 2  (t2  )FAR 3  (t3 ). 


Similarly,  for  A  =  3,  we  have 

FRR(t)  =  FRR 1  (?,  )FRR 2  (t2 )  +  AAA 2  (t2  )FRR 3  (t3 )  -  2FAA 1  (f,  )FRR 2  (t2  )FAA 3  (?3 ) 

=  FAA 1  (?,  )FAA 2  (f  2  )(1  -  FAA 3  (f  3 ))  +  AAA 1  (t,  )(1  -  FRR 2  (f  2  ))FAA 3  (t3 )  (34) 

+  (1  -  FRR 1  (t,  ))FRR 2  (t2 )FRR\t3 )  +  AAA 1  (f,  )FAA 2  (t2 )FAR \t3). 


If  the  integrated  system  FAR(t)  <  FAR '(/,),  then  we  have 


FAR1  (ti  )FAR 2  (t2 )  +  FAR 2  (t2  )AAA 3  (t3 )  -  2FAR 1  (t1  )FAR 2  (t2  )FAR 3  (t3 )  <  FAR 1  (t{ ) .  (35) 


The  following  equation  can  be  derived  from  (35), 


FAR 1  (q )  > 


FAR2(t2) 

1  +  (l  -  FAR 2  (t2  )(1  -  2 FAR 3  (f 3 ))/ FAR 3  (t3 ) ' 


(36) 


For  FAR(t)  <  FAR2  (t2),  we  have 
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(37) 


FAR2  (t2)> 


FAR 1  (tl ) 

1  +  (l  -  FAR 1  (t ~ )(1  -  2/u4R 3  (t  3  ))/iu4R 3  (? 3 )  ‘ 


For  FAR(t)<FAR3(t3), 


FAR 3 (t3 )  > 


FAR](tt) 

1  +  (l  -  FAR'  (t\)(\  -2FAR1  (t2))l FAR1  (t2)  ' 


(38) 


If  the  FRR  of  the  integrated  system  is  smaller  than  that  of  the  individual  classifier,  the 
following  conditions  should  be  satisfied. 


VoxFRR{t)<FRR\tx), 


FRR 1  (t1 )  > 


_ ™2(t2) 

1  +  (l  -  FRR  2  (f 2  )(1  -  2 FRR 3  (f  3  ))/FRR 3  (t3 ) ' 


For  FRR(t)  <  FRR2(t2),  we  have 


FRR2  (t2)> 


FRR]  (tt ) 

1  +  (l  -  FRR 1  (f !  )(1  -  2FRR 3  (f  3  ))/FRR 3  (f  3 ) ' 


For  FRR(t)<FRR3(t2), 


FRR3(t2)> 


FRR 1  (tl ) 

1  +  (l  -  FRR 1  (f !  )(1  -  2 FRR 2  (t2 ))/ FRR  2  (t2  )  ' 


(39) 


(40) 


(41) 
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5.4  Soft  decision  level  fusion 


For  simplicity,  we  discuss  integration  of  two  biometric  systems  denoted  by  lF1  andT'j, 
respectively.  The  integrated  system  is  denoted  by  T .  The  outputs  by  individual  systems  T,  and 
are  called  scores,  which  stand  for  the  probability  of  claimant  to  be  a  genuine  or  an  imposter. 
Then  for  any  fusion  strategies,  an  error  is  expressed  as  (14)  and  (15).  If  we  assume 
that  El  (tl )  <  E2  (t2 )  <  A  <  En  (tN ) ,  then  it  is  easily  known  it  is  sufficient  to  prove  that 
E{t)  <  Ex  {tx ) .  For  a  two-sensor  fusion,  if  Bayesian  rule  is  chosen,  then  this  means  that 


Decide  w0 ,  if  (Xx  ,X2)<=R 
Decide  wx ,  otherwise 


(42) 


where R  -  {(Xl5  X2)\Crp(Xl,  X2\w0)  >  Cap(Xx,  |wj )} .  Since  Tj  and  ^2  are  independent,  we 
have 


p(Xx,X2  w0)  =  pt(Xt  w0)p2(X2  w0), 


P(X i,X2  w0)  =  pt(Xt  w0)p2(X2  w0). 


(43) 


Then 


FAR{t)  =  1  -  f  p(Xl,X2\w0)dXldX2 

JR0  I 

= 1  “  Lo  P\  (x\\wo  )dx\\Rn  Pi  (xi  |w0  )dX2  (44) 

=1-(1-F^i(^1))a-F^2(^2)). 

It  can  be  easily  seen  from  (44)  that  FAR(t)>  FAR1  (tx)  and  FAR(t)  >  FAR 2 (t2 )  .  Thus  the  two¬ 
sensor  configuration  cannot  improve  the  false  acceptance  rate  by  the  Bayesian  decision  rule. 

From  the  definition  of  the  false  rejection  rate  and  with  the  help  of  the  independence  of  the 
individual  classifiers,  we  have 


DRDC  Ottawa  CR  2005-052 


33 


(45) 


FRR(t)  =  f  p(X,  ,X2\w1  )dX,dX2 

JR0  I 

=  jRn  Pi  (Xl\Wl  )dXl  |fl(i  Pi  (Xl\Wl  )dX  2 
=  FRRl(tl)FRR2(t2). 


From  (45),  it  is  obvious  that  FRR(t)  <  FRR 1  (ti )  and  FRR(t)  >  FRR  '  (t2)  ■  Thus  the  false  rejection 
rate  of  the  integrated  system  is  reduced  compared  to  individual  sub-classifiers. 

The  exact  way  as  to  how  the  integrated  system  performs  better  in  terms  of  the  smaller  total  error 
is  described  in  [81].  The  Bayesian  decision  rule  is  adopted  to  prove  that  the  error  is  minimized. 
That  is,  assume  that 


CrP(Xl1X2Wo)_>l 

CaP(XlXl\Wl) 


That  is 


CrPiix  1 

W0)p2(X2 

w0) 

CaPl(X  1 

wl)p2(x2 

Wj) 

which  is  equivalent  to 

CrPljX 2K)  Pl(X  lK) 
CaPi(xiWi)  Pi(x  \W\) 


(46) 


(47) 


(48) 


It  was  claimed  in  [81]  that  from  the  Bayesian  rule,  the  total  error  will  be  smaller  than  the  other 
rules.  This  claim  needs  further  theoretical  proof. 
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6.  Simulation  Results 


6.1  AND  rule 


The  first  experiment  shows  the  performance  of  the  AND  rule  for  fusing  two  biometrics.  Figure 
21(a)  compares  the  fused  result  with  that  of  the  individual  biometrics.  In  Figure  21(b),  region  with 
fusion  threshold  “1”  represents  the  region  in  which  performance  of  the  AND  rule  based  fusion  is 
better  than  all  individual  classifiers.  The  region  with  “-1”  represents  the  area  where  the  fused 
result  is  worse  than  the  best  individual  biometrics.  FRR  and  FAR  vary  from  0.05  (N=  0)  to  0.0  ( N 
=  30).  We  can  observe  that  the  fused  result  is  better  than  all  individual  biometrics  only  in  some 
regions  for  the  AND  rule  for  fusing  two  biometrics. 


(a) 


Figure  21.  AND  rule  for  fusing  two  biometrics 
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The  second  experiment  shows  the  performance  of  the  AND  rule  for  fusing  three  biometrics. 
Figure  22(a)  compares  the  fused  result  with  that  of  the  individual  biometrics.  In  Figure  22(b), 
region  with  fusion  threshold  “1”  represents  the  region  in  which  performance  of  the  AND  rule 
based  fusion  is  better  than  all  individual  classifiers.  The  region  with  “-1”  represents  the  area 
where  the  fused  result  is  worse  than  the  best  individual  biometrics.  The  FRR  and  FAR  of  three 
biometrics  were  changed  as  follows:  FARl:  [0,  0.05];  FRR1:  [0,  0.05];  FAR2:  [0,  0.04];  FRR2: 
[0,  0.04];  FAR3:  [0,  0.03];  FRR3:  [0,  0.03].  Even  in  this  case,  we  can  notice  that  the  fused  result 
is  better  than  all  the  three  individual  biometrics  only  in  some  areas. 


(a) 


Figure  22.  AND  rule  for  fusing  three  biometrics 
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6.2  OR  rule 


The  third  experiment  shows  the  performance  of  the  OR  rule  for  fusing  two  biometric  signals. 
Figure  23(a)  compares  the  fused  result  with  that  of  the  individual  biometrics.  In  Figure  23(b), 
region  with  fusion  threshold  “  1  ”  represents  the  region  in  which  performance  of  the  OR  rule  based 
fusion  is  better  than  all  individual  classifiers.  The  region  with  “-1”  represents  the  area  where  the 
fused  result  is  worse  than  the  best  individual  biometrics.  FRR  and  FAR  vary  from  0.05  (A  =  0)  to 
0.0  (N=  30).  We  can  observe  that  the  fused  result  is  better  than  all  individual  biometrics  only  in 
some  regions  for  the  OR  rule  for  fusing  two  biometrics. 


(a) 


Figure  23.  OR  rule  for  fusing  two  biometrics 
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The  fourth  experiment  shows  the  performance  of  the  OR  rule  for  fusing  three  biometric  signals. 
FRRs  and  FARs  of  the  three  biometrics  were  changed  as  follows:  FARl:  [0,  0.05];  FRR1:  [0, 
0.05];  FAR2:  [0,  0.04];  FRR2:  [0,  0.04];  FAR3:  [0,  0.03];  FRR3:  [0,  0.03],  We  can  see  that  the 
fused  result  is  better  than  all  individual  biometrics  only  in  some  areas  for  the  OR  rule  with  three 
biometrics. 


FRR:  maxFRR*i/N2 


FAR:  maxFAR*i/N1 


□.12 


]  0B 


+  0.06 


0.04 


_  U2 


(a) 


FRR:  maxFRR*i/N2 


0  0 


FAR:  maxFAR'i/NI 


(b) 


Figure  24.  OR  rule  for  fusing  three  biometrics 


38 


DRDC  Ottawa  CR  2005-052 


6.3  Majority  voting  rule 


The  fifth  experiment  shows  the  performance  of  the  Majority  Voting  Rule  for  fusing  three 
biometric  signals.  The  FRRs  and  FARs  of  the  three  biometrics  were  changed  as  follows:  FARl: 
[0,  0.05];  FRR1:  [0,  0.05];  FAR2:  [0,  0.04];  FRR2:  [0,  0.04];  FAR3:  [0,  0.03];  FRR3:  [0,  0.03], 
The  comparison  results  are  shown  in  Fig.  25.  It  is  apparent  from  the  figure  that  the  fused  result  is 
better  than  all  individual  biometrics  over  the  entire  region.  On  the  other  hand,  the  AND  and  OR 
based  fusion  were  better  over  only  a  portion  of  this  entire  region. 


(a) 


(b) 


Figure  25.  Majority  Voting  rule  for  fusing  three  biometrics 
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6.4  Soft  decision-level  fusion 


The  sixth  experiment  shows  the  FRR  and  FAR  of  the  Bayesian  soft  decision-level  fusion  of  two 
biometrics.  Figure  26(a)  shows  that  the  FRR  of  the  integrated  system  is  reduced  compared  to  the 
individual  classifiers  while  the  FAR  of  the  integrated  system  is  increased  as  shown  in  Figure 
26(b). 


(a) 


(b) 


Figure  26.  Bayesian  soft  decision-level  fusion  of  two  biometrics 
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7.  Conclusion 


This  report  reviewed  the  basic  structure  for  information  fusion  including  raw  data  level  fusion, 
feature  level  fusion  (feature  concatenation  and  classifier  fusion),  decision  level  fusion,  and  multi¬ 
level  integrated  fusion  for  dependent  and  independent  sources.  The  raw  data  level  fusion  is 
mainly  applied  to  the  fields  of  multi-sensor  fusion  in  the  literature  for  target-tracking,  navigation, 
and  image  fusion.  Feature  level  and  decision  level  fusions  are  more  popular  for  unimodal 
biometric  fusion  (dependent  sources)  and  some  are  applied  to  multi-modal  biometric  system 
(independent  sources).  Different  techniques  for  biometric  fusion  at  these  levels  were  reviewed 
and  briefly  explained  in  this  report. 

In  the  latter  part  of  this  report,  some  preliminary  performance  analyses  on  fusion  of  independent 
sources  were  presented.  It  was  shown  that  under  certain  conditions,  fusion  of  independent  sources 
at  the  decision  level  can  in  fact  improve  the  performance  from  the  total  error  point  of  view. 
Simple  rules  such  as  AND,  OR  and  majority  rules  were  employed.  Both  theoretical  and  simulated 
results  were  obtained.  It  was  also  shown  that  for  fusing  two  biometric  traits  the  FAR  and  FRR 
cannot  be  simultaneously  decreased.  Although  it  is  desirable  to  decrease  both  FAR  and  FRR,  this 
might  not  be  a  big  problem  since  for  many  applications,  either  FAR  or  FRR  is  of  particular 
emphasis. 
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